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Abstract. Statistical mechanics relies on the complete though probabilistic de- 
scription of a system in terms of all the microscopic variables. Its object is to 
derive therefrom static and dynamic properties involving some reduced set of 
variables. The elimination of the irrelevant variables is guided by the maximum 
entropy criterion, which produces the probability law carrying the least amount 
of information compatible with the relevant variables. This defines relevant en- 
tropies which measure the missing information (the disorder) associated with the 
sole variables retained in an incomplete description. Relevant entropies depend 
not only on the state of the system but also on the coarseness of its reduced 
description. Their use sheds light on questions such as the Second Law, both in 
equilibrium an in irreversible thermodynamics, the projection method of statis- 
tical mechanics, Boltzmann's -theorem or spin-echo experiment. 

1 Introduction 

The concept of entropy underlies both thermodynamics and statistical physics, 
but its subtleties and its multiple aspects make it difficult to grasp. Our initial 
motivation is to understand how the entropy of thermodynamics S t h is related 
to the entropy of statistical mechanics. The solution of this problem will turn 
out to require the introduction in statistical physics of several different entropies, 
which will enlighten some other questions. 

For systems at equilibrium, the situation is simple, and it is dealt with in many 
tutorial books. At the macroscopic scale, the state of a system at equilibrium is 
characterized by some set of thermostatic variables, such as the particle number, 
the energy and the volume for a fluid. (We distinguish the thermodynamics of 
equilibrium from the thermodynamics of irreversible processes by terming them 
as " thermostatics" and "thermodynamics" proper, respectively.) The entropy S t h 
is some function of these variables, a function that can be determined indirectly 
by experiments. At the microscopic scale, the same state is described by a density 
operator D having the Boltzmann-Gibbs exponential form. The von Neumann 
entropy associated to D, defined by 



S(D) = —kTiDlnD, (1.1) 

is then identified with the thermostatic entropy Sth', the Boltzmann constant 
k = 1.38 x 10~ 23 JK _1 entering (1.1) accounts for the conventional choice of 
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the kelvin as the unit of temperature in thermal physics. This identification is 
justified through derivation of the Laws of thermostatics — in particular the 
Second Law — from the Boltzmann-Gibbs distributions of statistical physics. 
Moreover the microscopic approach allows us in principle to evaluate Sth, which 
macroscopically is an empirical quantity. 

The identification of the entropy S t h of thermodynamics with the von Neu- 
mann entropy S(D) cannot, however, be extended to dynamical processes. Con- 
sider an isolated system off equilibrium. On the one hand, according to the Second 
Law, its thermodynamic entropy Sth i s larger in its final equilibrium state than 
in its initial state. If its evolution proceeds through states close to equilibrium, 
the Clausius-Duhem inequality expresses moreover that the time-derivative of 
Sth is non-negative. On the other hand, the microscopic evolution is governed (in 
the Schrodinger picture) by the Liouville-von Neumann equation for the density 
operator, 

ih^ = [H,D], (1.2) 

where H is the hamiltonian operator of the system. It follows from (1.1) and (1.2) 
that S(D) does not vary with time. This constancy, when compared to the Second 
Law, constitutes a modern form of the paradox of irreversibility (W. Thomson 
1874, J. Loschmidt 1876), which expresses a qualitative difference between mi- 
croscopic and macroscopic dynamics. The solution of this paradox requires of 
course the thermodynamic entropy S t h of non-equilibrium states to differ from 
the von Neumann entropy S (D) . 

Actually, we shall associate below (§6 and §7) various so-called relevant en- 
tropies with a given density operator D, and shall show (§10) that one of them 
can be identified with Sth- We shall more generally illustrate the utility of the idea 
of relevant entropy, by showing how it underlies the projection method (§8 and 
§9) and how it enlightens some questions of irreversibility in statistical physics 
(§11 and §12). As a preliminary step, we briefly review (§§2 to 5) some aspects of 
the concepts of state and of entropy in statistical physics, on which we shall rely 
in the following. In particular it will be important to distinguish the complete 
statistical description of a state (§2) from its incomplete descriptions (§5) and 
to grasp the meaning of the maximum entropy criterion (§4). Although this ar- 
ticle is self-contained, it is somewhat sketchy. Further explanations, details and 
developments can be found in the first few and last few chapters of reference [1]. 
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2 Complete statistical description: the density 
operator 

The state of a system at a given time can be defined in several ways that should 
not be confused. 

(i) Thermodynamics relies on the identification of some set of macroscopic 
quantities Ai, the control of which is sufficient to govern the whole macroscopic 
behaviour of the system. A state of this system is then characterized by speci- 
fying the quantities Ai, irrespective of the microscopic variables which otherwise 
remain arbitrary. At the microscopic level, the thermodynamic description is 
thus incomplete. 

(ii) At the other extreme, one can imagine states, termed as " micro states" , in 
which all the variables are defined inasmuch as possible. In classical mechanics, 
when the uncertainly principle may be disregarded, nothing prevents such a def- 
inition of the variables to be perfect: a microstate is characterized by specifying 
the positions and momenta of all the particles (within permutations in the labels 
of indistinguishable particles). In quantum mechanics, a microstate must retain 
probabilistic features; it is represented as a ket, a vector in the Hilbert space 
associated with the system. 

(iii) For large systems it is however impossible in practice to prepare a mi- 
crostate. It is thus necessary to resort to statistics. The definition of a state 
therefore refers, not to a single system, but to a typical system chosen out of a 
statistical ensemble of systems all prepared under similar conditions. We recall 
in the continuation of this section the main features of this last type of state. 

The quantum physical quantities, or "observables" , are represented as her- 
mitean operators A which act in the Hilbert space of the system. In the classical 
limit, an observable tends to a random variable which is a function of the (ran- 
dom) positions and momenta of the particles. Thus, in quantum mechanics, the 
observables play the role of non-commutative random variables. 

We characterize statistically a state by its density operator at the consid- 
ered time. This means that we can evaluate the expectation value (A) of any 
observable A at that time by means of the equation 

(A) = TrAD. (2.1) 

The description of the system furnished by its density operator D is thus sta- 
tistical, since in general the variance (A 2 ) — (A) 2 does not vanish, and complete, 
since any expectation value can be derived from (2.1). 

The density operator D is therefore the tool which implements the correspon- 
dence A i— > (A) from the whole set of observables A to their expectation values 
(A) . Actually, in classical statistical mechanics, D is replaced by an ordinary 
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probability distribution, from which the expectation value of any observable re- 
sults through an integration which replaces the trace of (2.1). Thus a density 
operator should be regarded as a probability distribution for non-commuting 
random variables. Conversely, a linear correspondence A i— > (A) defines a density 
operator D. The standard properties of D, normalization, hermiticity and posi- 
tivity, express that the expectation value of the unit operator I is (I) = 1, and 
that (A) is real, {A 2 ) is positive for any hermitean A. 

A density operator D is usually represented as a matrix D a p in the Hilbert 
space. However, it may be convenient (and it will be necessary in §8) to regard 
the two indices a,/3 of this density matrix as a single, compound index. The 
density operator is then represented, no longer as a matrix but as a vector in the 
so-called Liouville space, where the compound index a, (3 plays the role of a single 
set of coordinates. We can then perform linear transformations on the compound 
index. This defines Liouville representations of quantum statistical mechanics, 
which encompass the standard matrix representations but are more general and 
more flexible [2] [3] [4] [5] . In a Liouville representation, the observables are also 
represented as vectors with a compound index. They belong to the dual Liouville 
space, and an expectation value (2.1), written as 



(A) = Y,A a pD Pa = {A;D), (2.2) 

a/3 

appears as a scalar product between the two vectors A and D of the two conju- 
gate Liouville spaces. In a Liouville representation, the Liouville-von Neumann 
equation, which governs the evolution of D, takes the form 

§ =L D, (2.8) 

which is equivalent to (1.2). The liouvillian L is a superoperator, represented in 
the Liouville space as a matrix. In the Hilbert space, it appears as a tensor with 
2x2 indices, and (2.3) is written in terms of the density matrix D a p as 

= LaPrfsDs-y; (2.4) 

7/3 

comparison with the more standard notation (1.2) shows that 



L a/ 3, 7< 5 — (H a5 5f3 y — H 1 p8 a $) . (2.5) 

The simplest example of Liouville representation concerns the statistical state 
of a spin |. Rather than describing it by a 2 x 2 hermitean density matrix with 
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trace 1, it is handy to represent it by a vector in a 3-dimensional Liouville space, 
with merely the expectation values of the Pauli matrices as components. The 
equation of motion (2.3) then describes directly the Larmor precession of this 
vector in a plane perpendicular to the magnetic field. 

Another example of Liouville representation, the Wigner representation, is 
useful in the study of the classical limit of quantum statistical mechanics. In 
this limit, the observables tend to commuting random variables. For a system of 
TV indistinguishable particles, the density operator D in the Wigner representa- 
tion tends to an ordinary probability distribution in the 6iV-dimensional phase 
space of coordinates and momenta of these particles; it can be identified with the 
probability density in phase space of classical statistical mechanics. The traces 
in (1.1) and in (2.1) or equivalently the scalar product (2.2) reduce to integrals 
in phase space, with the measure 

N 

Ipr^ ViV! h 3N (2.6) 

i=i 

which arises from the summation over a and (3 of (2.2). The equation of a motion 
(2.3) directly yields the classical Liouville equation. 

3 The statistical entropy as missing information 

Information theory associates with an event n, which has the probability p n to 
occur, its surprisal I n = —k lnp n , a number which measures the amount of 
information that we gain when we get to know the occurrence of this event. 
Within the multiplicative constant k which defines the unit of information, this 
logarithmic expression for I n is imposed by the condition that the information 
is additive when knowledge is gained by steps. The surprisal vanishes when the 
occurrence of n is certain, it increases as its probability becomes weaker. We 
shall let k — 1 in the continuation. This will imply that Boltzmann's constant is 
replaced by 1, so that we take the joule as the unit of temperature. 

For a set of exclusive events n, one of which is expected to occur, the statistical 
entropy is then defined as the average amount of information which is gained when 
either event occurs, that is, 

S ({Pn}) = y]PrJn = -^PrMPn- (3.1) 

n n 

This expression, a function of the probabilities p n , equivalently measures the 
amount of information which is missing when only the probabilities p n of occur- 
rence of the various events n are known. 
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Returning to statistical mechanics, the eigenvalues p n of a density operator D 
may be interpreted as ordinary probabilities of the microstates which are the 
corresponding eigenvectors of D and which behave as exclusive events. The 
expression (3.1), rewritten as 

S (D) = —TrDlnD, (3.2) 

is identified with the von Neumann entropy (1.1). It is thus interpreted as the 
lack of information which arises from the incompleteness of our statistical de- 
scription by means of the density operator D. Like the density operator itself, 
the von Neumann entropy is a statistical quantity, since it is a natural measure of 
uncertainty. As such, it is not a property of the object at hand, but has partly a 
subjective nature linked to our knowledge of this object, which is extracted from 
a statistical ensemble. 

This subjective character is somewhat hidden in the alternative interpretation 
of S (D) as a measure of disorder. We can argue, however, that "disorder" and 
"missing information" are synonymous. Consider, for instance, the 13 spades 
taken from a pack of cards. A first configuration, in which they are set according 
to the decreasing values of bridge, ace, king, two, displays perfect order; its 
statistical entropy vanishes. Some second configuration, reached after a long 
shuffling, appears completely disordered. Nothing allows us to recognize any 
special feature in it. The reason for this difference is that we regard the former 
configuration as unique (with probability 1), while we regard the latter as just 
one among all 13! possible configurations. Before looking at it, we assign to it the 
probability of occurrence 1/13!, and the entropy is therefore lnl3!. However, if 
the shuffling is performed by a skilful conjurer who controls every card, he knows 
perfectly the arrangement in the second configuration, which for him is perfectly 
ordered. Indeed he can reshuffle it back to the first configuration. 

A similar situation occurs in spin-echo experiments. At the initial time, the 
spins are prepared in a perfectly ordered configuration: they all point in the 
x-direction. A permanent magnetic field is applied in the z-direction, but its 
magnitude is not quite the same at the site of each spin, so that the Larmor 
frequencies of the various spins are slightly dispersed around their average value. 
From the time on, the spins precess rapidly in the xy-plane, but at slightly 
different speeds, so that the total magnetization M decreases in length while 
rotating at the average Larmor frequency: it relaxes to zero. We thus reach after 
some time T a completely disordered configuration where the spins point towards 
arbitrary directions of the xy-plane. However, this configuration is not disordered 
for the spin-echo experimentalist, who knows where it was issued from. He takes 
advantage of this knowledge by applying to the system at the time T a brief 
pulse of magnetic field along x, which suddenly rotates each spin by ir, replacing 
it by its symmetric with respect to the x-axis. Thus, the spins which were ahead 
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in the precession are now behind and conversely. After a second delay T, the 
precession, which again takes place at different speeds according to the site of 
each spin, produces back the original ordered configuration. We shall return in 
§12 to this example which suggests that order, or entropy, is not defined uniquely 
in the seemingly disordered configuration attained after relaxation. 

4 The maximum entropy criterion 

Owing to its interpretation as the missing amount of information associated with 
the probability distribution p n , the statistical entropy (3.1) is currently used as a 
tool for statistical inference. The purpose of statistical inference is to make rea- 
sonable predictions about some quantities, starting from an exact or a statistical 
knowledge of some other quantities. We thus have to assign to the considered 
set of elementary events n a probability distribution p n , which should of course 
account for the available data but should otherwise be unbiased. 

It is natural to admit that, among two probability laws, the more biased 
one is the one which carries more information. This assumption leads us to 
assign the probabilities p n of each event by relying on the maximum entropy 
criterion, as was first advocated by Jaynes [6]. The knowledge of the available 
data, whether they are specified exactly or statistically, first sets up constraints 
on the probabilities p n . Then, among all probability laws compatible with these 
constraints, we select the one for which the statistical entropy (3.1) is largest. Any 
other probability law compatible with the data would have a smaller statistical 
entropy, i.e., would carry more information than the minimum amount needed to 
account for these data. Choosing probabilities by means of the maximum entropy 
criterion thus amounts to retain all the available information while discarding any 
other irrelevant information. 

This method has successfully been applied to numerous and varied problems, 
ranging from signal theory to image processing and from detection of astrophys- 
ical objects to determination of protein structures. Its extension to statistical 
mechanics relies on the maximization of the von Neumann entropy (3.2), a func- 
tion of D which measures the lack of information. A density operator is thereby 
assigned to a state which is characterized by some given conditions, by looking for 
the maximum of S (D) under constraints that account for these conditions. We 
shall work out this procedure in §6 and §7 after having shown (§5) some examples 
of such constraints. A direct justification of the maximum entropy criterion in 
statistical mechanics, based on Laplace's indifference principle (equal probabili- 
ties should be assigned when nothing in known) rather than on the existence of 
von Neumann's entropy, is given in ref. [7]. 
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5 Relevant observables and incomplete descrip- 
tions 



We have noted (§2,i) that the state of a system is characterized at the macroscopic 
level by some partial set of quantities A4. When only such an information is 
available, the microscopic description of the state is incomplete, since we know 
nothing about the other microscopic variables. We shall identify a macroscopic 
quantity Ai with the expectation value (Ai) of the corresponding observable Ai. 
We thus deal with a system for which only the expectation values (Ai) of some 
set of observables A^, that we refer to as the relevant set R = {Ai} , are specified. 
Such a partial knowledge, both statistical and incomplete, is not sufficient to 
characterize the density operator D, which is merely constrained to satisfy the 
set of equations 

Tr DAi = (Ai) = Ai. (5.1) 

Note that for some other quantities the correspondence from macrophysics 
to microphysics is more straightforward. For instance, if the particle number 
of a system is specified exactly, leaving no room for any statistical fluctuation, 
this particle number enters the microscopic description through the very defi- 
nition of the Hilbert space of the system. As another example, the volume Q 
of a fluid appears in the microscopic theory implicitly through the hamiltonian. 
Such quantities are accounted for directly in the microscopic theory, and not as 
expectation values (5.1) of observables. 

Let us illustrate the above correspondence between macroscopic quantities Ai 
and relevant observables Ai with a few examples. 

In thermostatics, the variables Ai are those which characterize the macro- 
scopic state in equilibrium. For instance the thermostatic state of a simple fluid, 
made of a single type of identical molecules enclosed in a vessel with volume ft, 
is characterized by two macroscopic data Ai, namely its particle number Af and 
its internal energy U. We wish to describe this system at the microscopic level in 
terms of a density operator D in the Fock space, that is, in the Hilbert space with 
an arbitrary number of particles. Our first datum is then the expectation value 
(N) = Af of the particle number operator (in the Fock space), while the internal 
energy U is identified with the expectation value (H) of the hamiltonian operator 
H. The set R of relevant observables contains here two elements, the operators 
iV and H . We noted above that the volume Q enters the problem through the 
expression of H, which contains a potential confining the particles within Q. 

Our identification of macroscopic data Ai with microscopic expectation values 
(Ai) raises a question, since the microscopic quantities, particle number and en- 
ergy, are allowed to fluctuate freely around their expectation values, in contrast 
to their macroscopic counterparts. However, the statistical fluctuations which 



8 



appear in statistical mechanics for macroscopic systems turn out to be extremely 
small (of order (N)^ 1 ^ 2 in relative value); they are actually much smaller than 
the experimental uncertainties on the variables Ai, so that this qualitative dis- 
crepancy between the macroscopic and the microscopic descriptions is ineffective. 

For other systems in thermostatic equilibrium, the set R of relevant observ- 
ables Ai can involve, apart from N and H, additional constants of the motion, 
represented by observables which commute with H. When some invariance is bro- 
ken, we have moreover to include order parameters, which also keep a fixed value 
in time at least on the time-scales of observation. 

For thermodynamic processes close to equilibrium, a state is described macro- 
scopically at each time by distinguishing within the system a set of subsystems 
a, each of which is nearly in local equilibrium [8]. The macroscopic state is then 
characterized by specifying the thermostatic variables for each subsystem. For 
instance, in the thermodynamic or hydrodynamic regime, a simple fluid can be 
analyzed into volume elements, larger than the mean free path, so as to have 
reached local equilibrium, but sufficiently small so as to be practically homoge- 
neous. The variables Ai a are then the number of particles in each volume element, 
their total energy and their total momentum (or equivalently the local densities 
of particles, energy and momentum, smoothed over the mean free path); the com- 
pound index ia denotes here both the location a in space and the nature i of the 
physical quantity. The relevant set R of observables A ia are the corresponding 
operators (or random variables in the classical limit). 

A similar analysis holds quite generally, for thermal, mechanical, chemical, 
electrical or magnetic processes, provided they are sufficiently slow so as to en- 
sure local equilibrium. The thermodynamic quantities are most often the locally 
conserved quantities, but may also include other variables varying slowly on the 
microscopic scale. 

For more rapid processes, these thermodynamic variables are not sufficient to 
determine the physics of the system; more detailed quantities should be included 
into the set Ai, although many microscopic variables are still kept aside. For 
instance, the description of a gas suited for the ballistic regime, or Boltzmann 
description, requires an analysis, not only in the ordinary space, but in the 6- 
dimensional single-particle phase space. The set of relevant variables Ai are then 
the densities / (r, p) of particles at each point of this space; the index i stands 
here for (r, p) . In the Boltzmann description, all the 2-point, 3-point, ... correla- 
tions are disregarded. At the microscopic level, the set R of relevant observables 
includes all the single-particle observables, but only these. The components of the 
density operator (or, in the classical limit, of the probability density in the 6N- 
dimensional phase space) referring to 2-particle, 3-particle... observables remain 
unspecified. 

Other examples of incomplete descriptions are provided, on the one hand by 
coarse- graining, on the other hand by collective variables. Consider for instance 
large nuclei. It is impossible to describe their state fully, but their dynamics can 
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be accounted for by following only some set of collective variables, such as the 
ones which describe the shape of the nucleus in a fission problem. 

The set of relevant observables always includes the quantities which can ex- 
perimentally be observed or controlled. However, the above examples show that 
for dynamical problems our description also includes other quantities, needed 
to account theoretically for the physics of the system. For instance, in fluid 
dynamics, the densities of particles, energy and momentum are not controlled 
everywhere, but they must be introduced to write the basic, Navier-Stokes and 
Fourier, equations. 

6 Least biased states and relevant entropies 

We return to inference in statistical mechanics, first focusing on the following 
single-time problem. Let R = {Ai} be some set of relevant observables. We 
assume that their expectation values (Ai) = Ai are specified at some given time, 
but that nothing else is known. From this incomplete information, we wish to 
make a reasonable statistical guess about the expectation value (B) of any other 
observable B at the considered time. Equivalently, we wish to construct at that 
time the density operator D which encompasses our whole statistical knowledge 
(§2), both within and outside R. 

To this aim, we use the maximum entropy criterion of §4. The density operator 
D to be assigned to the system should both fulfil the constraints (5.1) expressing 
that (Ai) = Ai and render the von Neumann entropy (3.2) maximum, so as 
to avoid including in D any additional information. The construction of the 
least biased state D is therefrom achieved through introduction of a lagrangian 
multiplier r ) i associated with each constraint (5.1). The result has the form 



where the term \l/ accounts for the normalization TrDo — 1 of Dq. The exponen- 
tial in (6.1) reflects the occurrence of a logarithm in S (D) . 
A direct proof of (6.1) relies on the inequality 



which holds for any pair of density operators D and D' . If in (6.2) we replace 
D' by (6.1) and D by any other density operator satisfying the constraints (5.1), 
the right-hand side reads 




(6.1) 



S(D) < -TrDlnZT, 



(6.2) 



TrD In D = V + (A t ) = -Tr A) In D = S (D ) , 



(6.3) 



10 



and we readily see that S (D ) > S (D) , as we anticipated. 

The determination of D is achieved by specifying the values of the parameters 
^ and 7j. The normalization condition, written as 



M>({ 7l }) = lnTrexp -YfliA 



(6.4) 



first defines \l/ as a function of the multipliers 7^. The latter quantities are then 
determined by inserting (6.1) into (5.1), which yields a set of equations, 



relating the multipliers r ) i and the macroscopic data (Ai) = Ai- 

The above procedure is standard in thermostatics. If for instance the relevant 
set {Ai} consists of the particle number operator N and the hamiltonian H, we 
find thus a statistical justification for the grand canonical density operator (6.1). 
The data A\ and A2 are Af and U, respectively, while the multipliers j 1 and 7 2 
are identified with —fi/kT and 1/kT, respectively, where T is the temperature 
(in kelvin) and [i is the chemical potential. The function (6.4) is then identified 
with the Massieu thermodynamic potential, and (6.3) with the usual entropy of 
thermostatics since (6.5) implies the same equation 



as in macroscopic physics. 

A similar formal structure, analogous to that of thermostatics, occurs for any 
imperfectly known state at a given time, when the expectation values (Ai) = Ai 
of some relevant set R = {Ai} of observables (as exemplified in §5) are the only 
available information. A multiplier r y i is associated with each given quantity Ai. 
The least biased state D (eq. 6.1) is characterized equivalently by the set {Ai} or 
by the set {7^} . The relations (6.3) between these conjugate variables {Ai} and 
{7j} are generated from the generalized thermodynamic potential ^ of eq.(6.4). 
The von Neumann entropy (6.3) of the state (6.1), when regarded as a function 
of the variables Ai, appears as the Legendre transform of \1/ ({7^}) . We denote it 
as 



and it alternatively generates the relations between the conjugate sets {Ai} and 
{li} through 



dS = 7l rfAT + l2 dU = -(dU- fidAf) 



(6.6) 




(6.7) 
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(6.8) 



We term Sr the relevant entropy associated with the set R = {Ai} of relevant 
observables. By construction it is the maximum value. S (-Do) of the uncertainty 
- as measured by the von Neumann entropy S (D) — of the various microscopic 
states D which are equivalent as regards the expectation values 

(A) = Tr A t D = Tr A t D = A l (6.9) 

of the relevant observables, but which are inequivalent as regards the irrelevant 
observables (those which lie outside R). The relevant entropy thus characterizes 
the randomness of an incomplete statistical description of the system, involving 
only the set Ai at the considered time: it measures the amount of information 
which is missing when only the quantities (Ai) = Ai are specified. While D 
carries no other information than these expectation values, the other density 
operators D satisfying (6.9) carry an extra amount of information which pertains 
to irrelevant observables and which is measured by 

S R ({Ai})-S(D). (6.10) 

The relevant entropy Sr decreases as the set R is enlarged. Consider a statis- 
tical ensemble of systems all prepared in the same manner, by controlling some 
set R n of observables. Imagine that we have no information at the start and that 
we determine better and better the state by learning the expectation values Ai for 
nested, larger and larger sets R\ C R2 C ... C R n . (Or imagine that conversely 
we begin with the full statistical knowledge of R n and discard information by 
steps, until we keep only information on R\.) Each step corresponds to a positive 
gain (or loss) of information. This is expressed by the inequalities 

S Rl > S R2 > ... > S Rn (6.11) 

for the relevant entropies associated with each set. 

An exercise on relevant entropies can be found in ref. [1], exerc. 3c. 

7 Reduction of the description 

The approach of §6 covers the foundations of thermostatics. New inference prob- 
lems arise, however, in non-equilibrium statistical mechanics. 
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We first have to assign a density operator D (to) to the system at the initial 
time t . Some macroscopic variables Ai are controlled externally so as to prepare 
the initial state, and this preparation is of course incomplete. We wish to infer 
therefrom the complete statistical description which is required to work out the 
microscopic approach of statistical physics. Such a question has been solved in §6, 
where we showed that the initial state D (t ) should have the form (6.1) in terms 
of the observables Ai under experimental control. Actually these observables 
which can be handled in practice to prepare the system may constitute only a 
subset of the chosen set R of relevant observables. In spite of that, we note that 
D (t ) = D (t ) is still expressed as (6.1) in terms of the full set R, provided 
we associate a vanishing multiplier 7^ (t ) = with any relevant observable Aj 
that is not controlled in the preparation. The values of the multipliers 7^ (to) 
associated with the controlled data Ai (to) are still determined by (6.5), while 
we infer from these controlled data the expectation values of the remaining, non 
controlled relevant observables by means of 

(7.1) 

7j =o 

The next problem is to make reasonable statistical predictions about any ob- 
servable B at any time t subsequent to t (or symmetrically to make retrodictions 
for times t earlier than t ). Starting from the initial condition D (t ) = -Do (^0) 
inferred above from the data Ai (to) , we determine D (t) (at least in principle 
if not in practice) by solving the Liouville-von Neumann equation (1.2). The 
required quantities are then obtained as 

(B) t = TrBD (t) . (7.2) 

The equation of motion (1.2) for D (t) entails the constancy of the von Neu- 
mann entropy (3.2), which keeps its initial value: 

S[D(t)] = S R [{A l (t )}]. (7.3) 

This equation expresses that the complete statistical description provided by D (t) 
conserves the information that was initially stored in the system through the 
controlled data Ai (to) ■ This conservation is natural, owing to the reversibility of 
the microscopic evolution. However, D (t) has no reason to keep the exponential 
form (6.1) that it had at the time to- Hence, with respect to the relevant set 
R, D (t) is generally biased: the information that it carries pertains not only to 
this set R (as at the initial time t = t ) but also to irrelevant observables. The 
evolution thus generates some transfer of information from relevant to irrelevant 
variables. 



A, (to) 



#7, 
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Let us make this point clearer by getting rid of the irrelevant information at 
each time. We thus wish to study the relevant quantities. 



A (t) = {Ai) t = TrAiD (t) , 



(7.4) 



irrespective of the other variables. The method of §6 allows us to discard the 
information carried by the latter, irrelevant variables. It provides us with a 
state D (t) , which accounts for the values (A) t given by (7.4) for the relevant 
quantities and which, contrary to D (t) , is unbiased with respect to the set R. 
The values of (B) t provided by Dq (t) are those inferred from the knowledge of the 
set Ai (t) only; they differ from the actual values (7.2) which are inferred from 
the initial conditions Ai (t ) and from the microscopic evolution. The missing 
information associated with the sole knowledge of the set (7.4) is the relevant 
entropy Sr [{Ai (t))} , i.e., the von Neumann entropy S[D (t)] of the unbiased 
state Dq (t) . 

We are thus led to introduce, beside the complete statistical microscopic de- 
scription through D (t) , a reduced description in terms of a simpler density op- 
erator D (t) which carries the sole information on the relevant set R. At each 
time, we associate with D (t) a reduced state Dq (t) which has the exponential 
form (6.1) and which satisfies the conditions 



The reduced state D (t) accompanies D (t) in its motion. The parameters j i (t) 
of (6.1) which determine D (t) are obtained from (7.5). The correspondence 
between the sets {7^ (t)} and {Ai (t)} is implemented by (6.5) or (6.8), so that 
we can characterize the reduced state equivalently by the set {Ai (t)} , by the set 
{7j (£)} or by the density operator D (t) . 

The information carried by this reduced description is smaller than that car- 
ried by the full description through D (t) . The difference 



where we used (7.3), measures the irreversibility with respect to the set R. It is 
the amount of information which has leaked from the relevant to the irrelevant 
set. It is also interpreted as the transfer of disorder from the irrelevant to the 
relevant set. According to (7.3), the total disorder S [D (t)} remains constant, 
while (7.6) shows that the disorder associated with the relevant set has increased 
between the times t and t. 

Altogether, we have been able to relate the macroscopic description in terms 
of the sole variables Ai (t) to the microscopic, detailed description of statistical 
mechanics. The following scheme was brought out: 



TrAA) (t) = TrAiD (t) = Ai (t) . 



(7.5) 



S [A) (*)] -S[D (t)} = S R \{A (t)}] - S R [{Ai (t )}] , 



(7.6) 
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{A (t )} => D (t ) <-> D (t) ^ {Ai (t)} D (t) . (7.7) 



From the data {Ai (to)} which are controlled at the initial time, we first inferred 
the state D (to) = D (to) at that time by means of the maximum entropy 
criterion. The solution of the Liouville-von Neumann equation, which is the basic 
dynamical equation in statistical mechanics, related D (t) to D (to) reversibly. We 
then found the relevant quantities {Ai (t)} through (7.4). Eventually we inferred 
D (t) from the latter set, using again the maximum entropy criterion. This 
last step will be used in §8 to eliminate explicitly the irrelevant variables. The 
introduction of -Do (t) has also been necessary to define the relevant entropy, 
which according to (7.6) characterizes the loss of information in the step D (i) \— > 
{Ai (t)} or the irreversibility of the correspondence from {Ai (to)} to {Ai (t)} . 

8 The projection method 

Macroscopic descriptions of dynamical processes rely on more or less phenomeno- 
logical equations for the macroscopic variables {Ai (t)} . We now consider a major 
problem of statistical physics, namely justifying these macroscopic equations from 
the microscopic, more fundamental theory. To this is aim, we have to get rid of 
the irrelevant microscopic variables so as to work out the scheme (7.7) and to find 
equations of motions for the set {Ai (t)} only. The projection method of Naka- 
jima and Zwanzig (4.5) achieves this goal, at least formally but quite generally. 
We sketch it here in order to shed light on the meaning of varied macroscopic 
dynamical equations. 

We first note that the knowledge of the time-dependence of the set {Ai (t)} 
is equivalent to that of the reduced state Dq (t) . Indeed, we can deduce Ai from 
-D through (7.5). Conversely D , as expressed by (6.1), (6.3), depends on time 
through the variables {7^} , which in turn are related to the variables {Ai} by 
(6.5) or (6.8). We shall in the following regard D as a function of the set {Ai} , 
and clS du function of time through this set. 

The time-dependence of the reduced state D that we thus seek for should be 
obtained from the Liouville-von Neumann equation of motion for the full D. We 
shall work out this derivation in a Liouville representation (§2), where both the 
density operators and the observables are regarded as vectors in two conjugate 
vector spaces. In such a representation the latter equation of motion as well as 
the correspondence from D to Do will be expressed by formally simple equations. 
On the one hand the Liouville-von Neumann equation is written as (2.3) in terms 
of the liouvillian superoperator L. 

On the other hand, D , which is a function of the set {Ai} through (6.1), (6.3) 
and (6.8), follows from D through the evaluation of the set {Ai} by means of (7.4). 
It can be shown that this construction amounts, in a Liouville representation 
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where D and D are regarded as vectors in the Liouville space, to a projection of 
D onto the surface (6.1): 



D = PD. (8.1) 

The maximum entropy criterion is thus implemented as a projection. In a Hilbert 
space representation, the projection superoperator P is a tensor with 2x2 indices, 
which acts in the same way (2.4) as the liouvillian on the density matrix D. The 
projection P depends on the variables {Ai\ . It acts on its left hand side upon 
the observables, and satisfies in particular 



A l P = A i , IP = I. (8.2) 

The projection thus keeps the relevant set R of observables invariant. These facts 
are easily checked from the expression of P, 

d D 

i ^ 

which however we shall not need below. Eq. (8.3) exhibits P as a sum of elemen- 
tary projections, each of which is the tensor product of a state-like vector and 
an observable-like vector, that lie in the conjugate Liouville spaces; the partial 
derivative dD /dAi is evaluated by regarding D as a function of the variables 
{A} • 

The complementary projection superoperator Q onto the irrelevant space is 
defined as 



Q = I - P, (8.4) 

where I is the unit superoperator in the Liouville space. The properties P 2 = 
P, Q 2 = Q and PQ = show that the Liouville formalism helps separating the 
relevant and the irrelevant components. The density operator D can thereby be 
split at each time into its reduced relevant part D Q and its irrelevant part D 1 
defined by 



D 1 = QD. (8.5) 

We wish to find the time-dependence of D from the Liouville-von Neumann 
equation (2.3) which deals with the full D. We take therefore the time-derivative 
of (8.1), using (2.3). This yields 
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(8.6) 



where the time-dependence of P occurs through the variables {Ai} . Apart from 
its last term where D\ is as get unknown, (8.6) is a differential equation for D 
or for the set {Ai} . It can be written equivalently, using (7.5), as the set 

dA 

-^p = (4;LA>) + (A; L-Di), (8.7) 

where we made use of the notation (2.2) for a scalar product in the Liouville 
space and where we took (8.2) into account. 

In order to eliminate D\ from (8.7), we write the equation of motion for Di, 
by taking the time- derivation of (8.5), which yields 



+ -jrDi - QLDi = QLD . (8.8) 
at at 

The equations (8.6) and (8.8) couple the projected parts D and Di of D. As 
(8.8) is linear in Di, with coefficients depending on D , it can be solved formally 
by introducing its Green's function W(t, t') . This quantity is a superoperator, 
which is the solution of 



+ -^)w(t,t>) = QS(t-t>), (8.9) 

with W(t,t') = for t < t! . Using the initial condition Di(t ) = that results 
from D (t ) = D (t ) (§7), we thus express D x in terms of D as 

Di (t) — [ dt'\N(t,t')LD (t'). (8.10) 

Insertion of (8.10) into (8.7) achieves one goal, finding a set of dynamical 
equations for the variables Ai (t) alone: 

dA- /"* 

= (A ; LA, (0 ) + (A ; jf dt'L\N(t,t')LD (t')), (8.11) 

where W was defined by (8.9). The elimination of the irrelevant variables has 
resulted in a set of integro- differential equations which are non-linear and non- 
markovian. Indeed the last of (8.11) does not depend on D (t) alone as does 
the first term, but on D at earlier times, directly through D (f) and indirectly 



17 



through the projections P and Q which enter the definition (8.9) of W. The 
evolution of the set {Ai} thus involves a memory, which is characterized by the 
memory kernel W (t, t') . The latter superoperator depends on the history of Ai 
between the times t' and t. 

This first term in (8.11) represents the direct coupling of the observables within 
the relevant set R; it involves the components PLP of the liouvillian. The last, 
retarded term is interpreted as the effect of an indirect coupling through the ir- 
relevant observables. Since W satisfies 

w(*,0 = Q(*)W(*,OQ(0> (8- 12 ) 

and since its definition (8.9) involves only the components QLQ of the liouvillian, 
it describes the time- evolution of the irrelevant set of variables that we have 
eliminated. This irrelevant set is coupled in (8.11) to the set R of interest, at the 
time t through the cross components PLQ of the liouvillian, at the earlier times 
t' through its components QLP. 

We noted that the von Neumann entropy S (D) , associated through (3.2) 
with the complete statistical description D, remains constant during the evolu- 
tion generated by (1.2), expressing that this evolution conserves information or 
disorder. We have here to associate with the reduced equation of motion (8.11) the 
relevant entropy Sr ({Ai}) expressed by (6.7), which characterizes the disorder 
in the relevant set R. Its time variation, obtained from (6.8) and (8.11) as 

dSfi \ dAi 

~~dT = ^ ll ^tT 

i 

= (-In £>„(*) 5 t dt'L\N(t,t')LD (t')),(8.13) (1) 

J to 

has no reason to vanish. In fact, the positivity of (7.6) shows that, at least 
between the times to an d t, the relevant entropy increases. The expression (8.13), 
after integration over time, relates this loss of information to the coupling of the 
relevant set R with the irrelevant observables. 

9 Short- memory approximation 

Our whole construction relied on the choice of the relevant set R, which guided by 
macroscopic physics, itself based on experiment and on phenomenological theory. 
The macroscopic dynamical equations for the variables Ai (t) usually have the 
form of differential equations. In contrast, the exact equations (8.11) have a 
different structure, involving memory effects for whatever choice of R. A further 
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step is thus needed to found the macroscopic equations upon the microscopic 
theory. 

We wish the last term of (8.11) to depend approximately only on the set 
{Ai (£)} at the considered time t, not at earlier times. This can be achieved if the 
choice of the relevant set R satisfies some conditions about time-scales. Actually, 
the main contribution to the integral over t! in (8.11) should arise from times t' 
sufficiently close to the upper bound t so that D (t) does not differ significantly 
from D (t) . This requires the weight 



P(t)LW(M')LP(t% 

which multiplies -Do (t') and which accounts for the dynamics of the irrelevant 
variables, to decrease sufficiently fast as t — t' increases. The relevant set should 
thus have been chosen by selecting the slowest variables. We understand the 
possibility of a short time-range r for the expression (9.1) by noting that it is 
equal to P (t) LQ (t) LP (t) when t = t', and that for t > t' it includes a sum of 
a large number of terms, associated with the evolution of the irrelevant degrees 
of freedom which oscillate rapidly and interfere destructively as t — t' increases. 
If (9.1) is thus negligible beyond the delay r, the memory in the evolution of 
the relevant set due to their coupling with the irrelevant ones is lost after the 
time-lapse r. 

We can then replace D (t 1 ) by D (t) in the equations (8.11), which become 
approximately 



dt 

we defined K (t) as 



dA 

' " [L+LK(f)L]£> (f)); (9.2) 



K(f) = f dt'\N(t,t'). (9.3) 

J t 

The lower bound to m (9-2) is not significant since the integrand is negligible 
outside the interval t > t' > t — r. The superoperator K acts only on the irrele- 
vant space since K (t) = Q (t) K (t) Q (t) , as seen from (8.12) and from the slow 
variation of Q (t) on the scale r. The bracket in (9.2) can equivalently be replaced 
by P [L + LKL] P, which shows that it acts on the relevant space only. 

The projection method, completed by this short-memory approximation, has 
thus led to the equations of motion (9.2) which provide a microscopic basis for 
the macroscopic dynamics. The possibility of describing the system by differen- 
tial equations for some partial set of relevant variables, without considering the 
remaining irrelevant microscopic variables, relied on the recognition of different 
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characteristic time-scales for the two sets. The influence on the macroscopic dy- 
namics of the irrelevant variables which have been eliminated then occurs through 
the factor PLKLP in (9.2). The method is general and flexible, as examples will 
show. It is however formal, since the solution of the equation (8.9) for W is at 
least as difficult as that of the Liouville-von Neumann equation. Further approx- 
imation (such as perturbative expansions) specific to the considered problem, 
are therefore needed to evaluate K from first principles. A rough approximation 
consists in estimating only the memory-time r and in taking K ~ rQ. 

When the approximation (9.2) is valid, the dissipation (8.13) has the instan- 
taneous form 



-^ = (-hiD ;lKLD ). (9.4) 

It is positive at each time, expressing a continuous flow of disorder from the irrel- 
evant to the relevant variables. Although dissipation appears from (8.13) to be a 
memory effect, it is in fact a short-memory effect, taking place on the time-scale 
t. The reconciliation of microscopic reversibility with macroscopic irreversibility 
thus requires the existence of two different time-scales. 



10 The thermodynamic entropy 

We have seen in §5 that the macroscopic state of a system evolving in a thermo- 
dynamic regime is characterized by a set of local variables Ai a associated, for each 
subsystem located at a with some or other quantity i. These quantities may flow 
from a subsystem a to another, and thermodynamics deals with such a transport 
[8]: thermal conduction corresponds to flow of energy, diffusion or electric current 
to flow of particles, fluid dynamics to flow of momentum. The thermodynamic 
equations provide the time-dependence of the variables Ai a - They can be decom- 
posed into: (i) Conservation laws relating the time- derivatives dAi a /dt to fluxes 
from each subsystem to its neighbours, (ii) For each subsystem a, relations be- 
tween the variables Ai a and their conjugate local intensive variables 7 - , such as 
temperature, chemical potential or velocity; these relations are at each point and 
each time the same as in thermostatics, (iii) Definition of affinities as differences 
lia ~ lib or as gradients of the intensive variables, (iv) Responses of the fluxes 
to the affinities. 

The equations for the set {^4« a } characterize the exchanges which occur be- 
tween the subsystems. Such exchanges take place on a time-scale which is large 
compared the time-scale r of the microscopic processes. In the present case, the 
latter time r corresponds to the delay after which the effect of the perturbation 
of a microscopic variable has been forgotten. It is thus the relaxation time which 
brings the system to local equilibrium. Thereafter the evolution slows down, and 
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proceeds in the thermodynamic or hydrodynamic regime where the dynamics 
involves the local quantities A{ a only. In a gas, r is of the order of the delay 
between two successive collisions of a particle. 

The occurence of a larger time-scale for the thermodynamic variables Ai a is 
related to the fact that these variables are conserved or nearly conserved. Indeed, 
the conservation laws imply that Ai a can increase only if A^ in a neighbouring 
subsystem b decreases, under the effect of a flux from b to a. This requires a 
coupling between the subsystems a and b. However, due to the macroscopic size 
of the subsystems, such a coupling is an interface effect which may be relatively 
weak; this hinders the fluxes and hence the time-variation of the variables A\ a - 

The microscopic incomplete description associated with the set of thermody- 
namic variables Ai a is based on a reduced density operator D of the form (6.1). 
Separation of the observables Ai a according to the subsystem a that they refer 
to expresses D as a tensor product 



D = Y[D a , D a = exp 



(10.1) 



of elementary density operators D a , each of which refers to one of the subsystems. 
In fact D a has exactly the same form as the density operator of the system a 
alone in thermostatic equilibrium. For each subsystem the relations between the 
variables Ai a and 7^ are thus the same as in thermostatics. We can therefore 
identify the multipliers j ia of (10.1) with the corresponding intensive variables of 
thermodynamics. We can also identify the set (9.2) with the thermodynamical 
equations of motion. 

Moreover the factorization (10.1) of D Q implies that the relevant entropy (6.7) 
associated with the thermodynamic variables is simply the sum 



S R ({A ia })=Y,S(D a ) (10.2) 

a 

of the von Neumann entropies S (D a ) , each of which is equal to the thermo- 
static entropy of the subsystem a. This allows us to identify the relevant entropy 
Sr ({Ai a }) relative to the thermodynamic variables with the entropy of ther- 
modynamics. 

The Second Law of thermostatics refers to a system in local equilibrium at 
the initial time t ? i n global equilibrium at a final time t±. In the interval, the 
evolution does not need to take place in the thermodynamic regime. We may for 
instance deal with an explosive chemical reaction, or a shock wave, and the short- 
memory approximation of §9 is thus not necessarily valid. However, while the 
von Neumann entropy S [D (t)} remains constant, the positivity of (7.6) implies 
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that Sr [{Aia (h)}] > Sr [{Aia (t )}] , which proves microscopically the Second 
Law. 

If the evolution takes place in the thermodynamic regime according to (9.2), 
the dissipation (9.4) is at each time positive, which means that Sth keeps increas- 
ing. This property is the dynamical form of the Second Law, expressed by the 
Clausius-Duhem inequality 

— — ^ 0. 10.3 

11 The Boltzmann entropy 

We turn to the Boltzmann description of a gas (§5). Here the relevant observables 
Ai are the single-particle random variables; their expectation values are the set 
{Ai} = f (r, p) , i.e., the density of particles in the single-particle phase space. 
This description is more detailed than the thermodynamic description of the gas, 
which would rely only on the densities of particles, of energy and of momentum in 
the ordinary space r. It applies to more general regimes than the thermodynamic 
regime. In particular, in the ballistic regime, the time between collisions which 
characterizes the thermalization is larger than the time during which / (r, p) 
changes significantly; the mean free path is larger than the distance over which 
/ changes. No local temperature, no hydrodynamic flow can then be defined. 

While / (r, p) plays the role of the macroscopic variables {Ai} , the irrelevant 
variables are here the correlations between the particles of the gas. Such cor- 
relations are created by collisions; the characteristic time associated with their 
dynamics is the duration of a collision. Over time-scales larger than this very 
brief duration, and over distances larger than the range of the interactions, the 
dynamics of / (r, p;t) is governed by the semi-phenomenological Boltzmann equa- 
tion 



= - P. V, / + ![/]. (11.1) 
at m 

This equation of motion for a gas is adequate under conditions much more gen- 
eral than the equations of thermodynamics. Like the latter ones it should be 
supplemented by boundary conditions accounting for the vessel in which the gas 
is enclosed. We shall consider it as a macroscopic equation and / (r, p) as a con- 
tinuous set of macroscopic variables, whereas the Liouville equation for the full 
D is the exact microscopic equation. The last term J [/] is the collision integral, 
a quadratic functional of / involving integration over momenta. 
By introducing the quantity 
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H(t)= d 3 rd s pf\nf, 



(11.2) 



Boltzmann proved that his equation (11.1) satisfies the H-theorem, namely 



§<o. (n.3) 

This exhibits the irreversibility of the Boltzmann equation, which contrasts with 
the reversibility of the more basic Liouville equation. 

At the microscopic level the Boltzmann description relies on a reduced density 
Dq in the 6iV-dimensional phase space, which is the classical equivalent of (6.1). 
The random variables Ai entering it are the classical single-particle quantities, 
so that -D describes an uncorrelated state. The projection (8.1) amounts here 
to chop off the correlations from D, while keeping the expectation values / (r, p) 
of single-particle quantities invariant. The Boltzmann equation is then identified 
with the result (9.2) of the projection method, the short-memory approximation 
being here justified by the short duration of the collisions. The two terms of 
(11.1) actually correspond to the two terms of (9.2). 

The relevant entropy associated with the Boltzmann variables / (r, p) is found 
by taking the classical limit of (6.4), (6.5), (6.7), where the trace is replaced by 
the measure (2.6). We obtain 



S B = S R (f) = |rf 3 rrf 3 p/(r,p)[l-ln^/(r,p)]. (11.4) 

This expression defines the Boltzmann entropy Sb, which is relevant for the dy- 
namics of a gas in whatever regime, provided the range of the interparticle po- 
tential is short compared to the distances between particles. Since the integral 
of / is the particle number, the Boltzmann entropy 5b is proportional to Boltz- 
mann's H, within the sign and within an additive constant. Thus the iJ-theorem 
expresses merely the increase (9.4) of the Boltzmann entropy. 

The if -theorem should not be confused with the Second Law. On the one 
hand, it applies only to gases. On the other hand, it covers much more general sit- 
uations, since in thermostatics the Second Law requires local equilibrium for the 
initial state, global equilibrium for the final state, and since in thermodynamics 
the Clausius-Duhem inequality requires local equilibrium at all times. 

The thermodynamics of gases is recovered from the Boltzmann description in 
regimes for which / varies slowly in space on the scale of the mean free path, 
slowly in time on the scale of the delay between collisions. In this limit, it can 
be shown that of / (r, p,t) does not differ much from a single-particle density in 
phase space having in terms of p the maxwellian form 
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fo ( r , P,*) = exp 



P 2 



(11.5) 



where 71,72 and 73 depend on r and t. The latter quantities are interpreted as 
the local intensive variables in the local equilibrium regime of thermodynamics, 
for instance l/&7 2 as the local temperature (the energy of the gas includes prac- 
tically the kinetic energy only of the particles). The Boltzmann entropy (11.4) 
reduces to the thermodynamic entropy S t h, the if -theorem to the thermodynamic 
dissipation (10.3). The thermodynamic equations of motion can also be obtained 
from the Boltzmann equation (11.1). The Chapman-Enskog method, on which 
this derivation is based, appears as an application of the projection method. Here 
/ is regarded as the set of microscopic variables. We thus replace in the general 
formalism D by /, and the Liouville-von Neumann equation by the Boltzmann 
equation (see for instance [1], chap. 15). Like D in eqs. (8.1) and (8.5), / is split 
into / = fo + fi, where fo, of the form (11.5), is obtained from / by means of a 
projection in the space of the functions of p. The elimination of fi is made here 
feasible by regarding fi as small compared to f in the considered regime, and it 
results in the Navier-Stokes and Fourier equations. 



12 Conclusion: the multiplicity of entropies 

The above example of a gas exhibits the occurrence in non-equilibrium statistical 
mechanics of several different entropies for a single system. 

(i) The von Neumann entropy S (D) defined by (3.2) is associated with the 
complete statistical description of the gas. It accounts in particular for the order 
carried by the correlations between its particles, which develop as collisions take 
place, even if they are absent at the initial time. This entropy remains constant, 
expressing the reversibility of the microscopic evolution and the conservation of 
the initial information. 

(ii) The Boltzmann entropy Sb defined by (11.4) is the relevant entropy as- 
sociated with the single-particle properties only, in an incomplete description 
disregarding all the correlations. Its increase in time, the ff-theorem, expresses 
the irreversible loss of information, or of order, which take place from the single- 
particle quantities towards the correlations. Actually, although the interactions 
an essential in the dynamics of the gas, the correlations which may exist between 
a particle and other ones are not effective when this particle undergoes a new col- 
lision; this is Boltzmann's Stosszahlansatz, that he used to justify his equation 
(11.1). 

(iii) The thermodynamic entropy S t h defined by (10.2) is the relevant entropy 
associated with only the locally conserved densities of particles, of energy and 
of momentum at each point. In the thermodynamic regime where / has the 
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form (11.5), the short-memory approximation is valid and it ensures that S t h 
continuously increases. This expresses the dissipation due to viscosity and to 
thermal conduction. 

(iv) The thermostatic entropy S cq (Af,U) defined in §6 can also be regarded 
as the relevant entropy associated with the total particle number and the total 
energy. These quantities are the globally conserved ones; momentum is locally 
conserved but not globally due to the collisions onto the walls, it vanishes at 
equilibrium. This definition holds at any time for a gas off equilibrium, with M 
and U keeping their initial values. However S cq , which does not vary, has no 
physical interest, except after the very long, global relaxation time has elapsed. 
The gas has then been brought to thermostatic equilibrium through the evolution 
generated by the Boltzmann equation. 

Because the above four descriptions are more and more incomplete, their three 
relevant entropies satisfy the inequalitites (6.11), to wit, 



The entropies S^h and Sb thus vary between the constant lower and upper bounds 
S (D) and S eq , and Sb never decreases. After local equilibrium is established, Sb 
and Sth are nearly equal and both tend to S cq when the system reaches global 
equilibrium; as we just saw, S eq is of interest only afterwards. Likewise, S t h has 
no interest before local equilibrium is reached, that is, during some time-lapse of 
the order of the delay between collisions. In this initial fast regime, the thermo- 
dynamic variables are inadequate. They obey only an impracticable equation of 
the type (8.11) including retardation effects, and not yet a thermodynamic equa- 
tion of the type (9.2). Accordingly, nothing forces dS t h/dt to be always positive 
at this stage of the evolution. 

The contrast between the constancy of S (D) and the increase of Sb is bet- 
ter understood by introducing still other relevant entropies [9]. Starting from 
the Boltzmann description based on single-particle quantities only, a hierarchy of 
more and more detailed descriptions are introduced by including two-particle cor- 
relations, then three-particle correlations, and so on. The sequence of associated 
relevant entropies S 2 , S 3 , ... satisfies the inequalities 



For an independent-particle initial state, all these entropies are equal at the initial 
time to- As Sb does, any S n begins to be equal to S (D) before increasing so as to 
reach eventually S cq when global equilibrium is attained. The curves S n (t) are 
ordered according to (12.2). As n increases, the rise of S n takes place later and 
later. This is necessary because, at any fixed time t, S n decreases with n down 
to S (D) , which is attained when n is so large that all the correlations created 



S cq > S th >S B > S(D). 



(12.1) 



Sb > S 2 > S 3 > ••■ > S(D). 



(12.2) 
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by the evolution from to to t are taken into account. For t and n both large, 
S n (t) behaves non-uniformly: it tends to S cq when t — > oo for fixed £. These 
two regimes correspond to t ^> nr and to t <C nr, respectively, where r is the 
time between successive collisions, since each new collision adds an additional 
correlation to a cluster already correlated by the previous collisions. 

Similar ideas apply to systems other than gases. In particular the adequacy 
of a partial description based on some choice of the set R relies on the absence 
of significant memory effects in the resulting equations of motion (8.11). If such 
effects occur, there exist within the set that we regarded as irrelevant some ob- 
servables which are coupled to those that we included in R and which have a 
comparable characteristic time. A decrease of the relevant entropy indicates that 
information, or order, has been stored in such hidden variables, and is released 
later on towards the variables retained in our description. Any hidden variable 
of this type should therefore be included into the relevant set R, so as to ensure 
that the information flowing towards the discarded variables is irretrievably lost. 

We illustrate this point by reconsidering the spin-echo experiment described 
in the end of §3. Let us first forget about the experimental possibility of rotating 
the spins by 7r around the x-axis at the time T by means of a pulse along this axis. 
The thermodynamics of spins is characterized by an entropy S t h (M) which is the 
relevant entropy associated with the macroscopic quantity, the magnetization 
M. It vanishes when all the spins are aligned, and increases as M decreases. 
Thus the relaxation of M, which can be described by a macroscopic equation of 
motion involving both rotation and damping, results as expected in the increase 
of 5th- However, the magnetic pulse at the time T along the a;- axis not only 
changes M but also changes the individual spins in a manner which keeps track 
of their history between the time and the time T. Owing to this possibility 
of manipulation, the memory about the initial state is not lost at the time T 
although the spins point in any direction of the plane. This apparent disorder 
of the spins reflects in fact the weak dispersion in the values at each site of the 
permanent field along z, since the spins have rotated between the time and 
the time T by an angle which is proportional to the local field. Due to this 
correlation between spins and fields, the total magnetization no longer follows 
between the times T and IT the macroscopic relaxation equation, but an equation 
involving memory about the individual spin sites. The decrease between the 
times T and 2T of the entropy S t h (M) down to its initial value zero reflects the 
possibility of retrieval of the initial information through this memory. We must 
thus include within the relevant set after the time T the hidden variables, i.e., 
the expectation values of the individual spins instead of that of the total spin. 
This produces a set of differential equations without memory, valid at all times. 
The relevant entropy associated with the individual spins never decreases and is 
now meaningful, contrary to the thermodynamic entropy S t h (M) . 
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